agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH v20 6/8] Row pattern recognition patch (docs). 513+ messages / 2 participants [nested] [flat]
* [PATCH v20 6/8] Row pattern recognition patch (docs). @ 2024-05-24 02:26 Tatsuo Ishii <ishii@postgresql.org> 0 siblings, 0 replies; 513+ messages in thread From: Tatsuo Ishii @ 2024-05-24 02:26 UTC (permalink / raw) --- doc/src/sgml/advanced.sgml | 82 ++++++++++++++++++++++++++++++++++++ doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++ doc/src/sgml/ref/select.sgml | 38 ++++++++++++++++- 3 files changed, 172 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml index 755c9f1485..b0b1d1c51e 100644 --- a/doc/src/sgml/advanced.sgml +++ b/doc/src/sgml/advanced.sgml @@ -537,6 +537,88 @@ WHERE pos < 3; <literal>rank</literal> less than 3. </para> + <para> + Row pattern common syntax can be used to perform row pattern recognition + in a query. The row pattern common syntax includes two sub + clauses: <literal>DEFINE</literal> + and <literal>PATTERN</literal>. <literal>DEFINE</literal> defines + definition variables along with an expression. The expression must be a + logical expression, which means it must + return <literal>TRUE</literal>, <literal>FALSE</literal> + or <literal>NULL</literal>. The expression may comprise column references + and functions. Window functions, aggregate functions and subqueries are + not allowed. An example of <literal>DEFINE</literal> is as follows. + +<programlisting> +DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +</programlisting> + + Note that <function>PREV</function> returns the price column in the + previous row if it's called in a context of row pattern recognition. Thus in + the second line the definition variable "UP" is <literal>TRUE</literal> + when the price column in the current row is greater than the price column + in the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when + the price column in the current row is lower than the price column in the + previous row. + </para> + <para> + Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be + used. <literal>PATTERN</literal> defines a sequence of rows that satisfies + certain conditions. For example following <literal>PATTERN</literal> + defines that a row starts with the condition "LOWPRICE", then one or more + rows satisfy "UP" and finally one or more rows satisfy "DOWN". Note that + "+" means one or more matches. Also you can use "*", which means zero or + more matches. If a sequence of rows which satisfies the PATTERN is found, + in the starting row of the sequence of rows all window functions and + aggregates are shown in the target list. Note that aggregations only look + into the matched rows, rather than whole frame. On the second or + subsequent rows all window functions are NULL. Aggregates are NULL or 0 + (count case) depending on its aggregation definition. For rows that do not + match on the PATTERN, all window functions and aggregates are shown AS + NULL too, except count showing 0. This is because the rows do not match, + thus they are in an empty frame. Example of a <literal>SELECT</literal> + using the <literal>DEFINE</literal> and <literal>PATTERN</literal> clause + is as follows. + +<programlisting> +SELECT company, tdate, price, + first_value(price) OVER w, + max(price) OVER w, + count(price) OVER w +FROM stock + WINDOW w AS ( + PARTITION BY company + ORDER BY tdate + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP PAST LAST ROW + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +</programlisting> +<screen> + company | tdate | price | first_value | max | count +----------+------------+-------+-------------+-----+------- + company1 | 2023-07-01 | 100 | 100 | 200 | 4 + company1 | 2023-07-02 | 200 | | | 0 + company1 | 2023-07-03 | 150 | | | 0 + company1 | 2023-07-04 | 140 | | | 0 + company1 | 2023-07-05 | 150 | | | 0 + company1 | 2023-07-06 | 90 | 90 | 130 | 4 + company1 | 2023-07-07 | 110 | | | 0 + company1 | 2023-07-08 | 130 | | | 0 + company1 | 2023-07-09 | 120 | | | 0 + company1 | 2023-07-10 | 130 | | | 0 +(10 rows) +</screen> + </para> + <para> When a query involves multiple window functions, it is possible to write out each one with a separate <literal>OVER</literal> clause, but this is diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml index 17c44bc338..8dbab31300 100644 --- a/doc/src/sgml/func.sgml +++ b/doc/src/sgml/func.sgml @@ -23124,6 +23124,7 @@ SELECT count(*) FROM sometable; returns <literal>NULL</literal> if there is no such row. </para></entry> </row> + </tbody> </tgroup> </table> @@ -23163,6 +23164,59 @@ SELECT count(*) FROM sometable; Other frame specifications can be used to obtain other effects. </para> + <para> + Row pattern recognition navigation functions are listed in + <xref linkend="functions-rpr-navigation-table"/>. These functions + can be used to describe DEFINE clause of Row pattern recognition. + </para> + + <table id="functions-rpr-navigation-table"> + <title>Row Pattern Navigation Functions</title> + <tgroup cols="1"> + <thead> + <row> + <entry role="func_table_entry"><para role="func_signature"> + Function + </para> + <para> + Description + </para></entry> + </row> + </thead> + + <tbody> + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>prev</primary> + </indexterm> + <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the previous row; + returns NULL if there is no previous row in the window frame. + </para></entry> + </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>next</primary> + </indexterm> + <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> ) + <returnvalue>anyelement</returnvalue> + </para> + <para> + Returns the column value at the next row; + returns NULL if there is no next row in the window frame. + </para></entry> + </row> + + </tbody> + </tgroup> + </table> + <note> <para> The SQL standard defines a <literal>RESPECT NULLS</literal> or diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml index 066aed44e6..8f18718d58 100644 --- a/doc/src/sgml/ref/select.sgml +++ b/doc/src/sgml/ref/select.sgml @@ -969,8 +969,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl The <replaceable class="parameter">frame_clause</replaceable> can be one of <synopsis> -{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] -{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] +{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] +{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax] </synopsis> where <replaceable>frame_start</replaceable> @@ -1077,6 +1077,40 @@ EXCLUDE NO OTHERS a given peer group will be in the frame or excluded from it. </para> + <para> + The + optional <replaceable class="parameter">row_pattern_common_syntax</replaceable> + defines the <firstterm>row pattern recognition condition</firstterm> for + this + window. <replaceable class="parameter">row_pattern_common_syntax</replaceable> + includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST + ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls + how to proceed to next row position after a match + found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the + default) next row position is next to the last row of previous match. On + the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next + row position is always next to the last row of previous + match. <literal>DEFINE</literal> defines definition variables along with a + boolean expression. <literal>PATTERN</literal> defines a sequence of rows + that satisfies certain conditions using variables defined + in <literal>DEFINE</literal> clause. If the variable is not defined in + the <literal>DEFINE</literal> clause, it is implicitly assumed + following is defined in the <literal>DEFINE</literal> clause. + +<synopsis> +<literal>variable_name</literal> AS TRUE +</synopsis> + + Note that the maximu number of variables defined + in <literal>DEFINE</literal> clause is 26. + +<synopsis> +[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ] +PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...] +DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...] +</synopsis> + </para> + <para> The purpose of a <literal>WINDOW</literal> clause is to specify the behavior of <firstterm>window functions</firstterm> appearing in the query's -- 2.25.1 ----Next_Part(Fri_May_24_11_39_19_2024_763)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="v20-0007-Row-pattern-recognition-patch-tests.patch" ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
* [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. @ 2026-06-02 09:37 Andy Fan <zhihuifan1213@163.com> 0 siblings, 0 replies; 513+ messages in thread From: Andy Fan @ 2026-06-02 09:37 UTC (permalink / raw) --- src/backend/access/common/printtup.c | 8 ++- src/backend/utils/adt/date.c | 32 +++++++---- src/backend/utils/adt/float.c | 62 ++++++++++++++++----- src/backend/utils/adt/int.c | 82 +++++++++++++++------------- src/backend/utils/adt/int8.c | 40 ++++++++++---- src/backend/utils/adt/numeric.c | 46 ++++++++++++---- src/backend/utils/adt/timestamp.c | 50 ++++++++++++----- src/backend/utils/adt/varlena.c | 52 ++++++++---------- src/include/libpq/pqformat.h | 61 +++++++++++++++++++++ src/include/utils/builtins.h | 24 ++++++++ 10 files changed, 325 insertions(+), 132 deletions(-) diff --git a/src/backend/access/common/printtup.c b/src/backend/access/common/printtup.c index 2e3eb8f56d3..ab625736ac1 100644 --- a/src/backend/access/common/printtup.c +++ b/src/backend/access/common/printtup.c @@ -393,17 +393,19 @@ printtup(TupleTableSlot *slot, DestReceiver *self) else { /* Binary output */ + Datum outputdatum; bytea *outputbytes; - outputbytes = DatumGetByteaP(FunctionCallInvoke(thisState->outstate)); + outputdatum = FunctionCallInvoke(thisState->outstate); Assert(!thisState->outstate->isnull); /* - * If outputbytes == NULL, the send function directly appended a + * If outputdatum == 0, the send function directly appended a * correctly formatted message. */ - if (outputbytes) + if (outputdatum) { + outputbytes = DatumGetByteaP(outputdatum); pq_sendint32(buf, VARSIZE(outputbytes) - VARHDRSZ); pq_sendbytes(buf, VARDATA(outputbytes), VARSIZE(outputbytes) - VARHDRSZ); diff --git a/src/backend/utils/adt/date.c b/src/backend/utils/adt/date.c index 7f746dd84c9..43284d43fda 100644 --- a/src/backend/utils/adt/date.c +++ b/src/backend/utils/adt/date.c @@ -180,7 +180,6 @@ Datum date_out(PG_FUNCTION_ARGS) { DateADT date = PG_GETARG_DATEADT(0); - char *result; struct pg_tm tt, *tm = &tt; char buf[MAXDATELEN + 1]; @@ -194,8 +193,10 @@ date_out(PG_FUNCTION_ARGS) EncodeDateOnly(tm, DateStyle, buf); } - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1606,7 +1607,6 @@ Datum time_out(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -1615,8 +1615,10 @@ time_out(PG_FUNCTION_ARGS) time2tm(time, tm, &fsec); EncodeTimeOnly(tm, fsec, false, 0, DateStyle, buf); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -1652,11 +1654,21 @@ Datum time_send(PG_FUNCTION_ARGS) { TimeADT time = PG_GETARG_TIMEADT(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, time); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, time); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, time); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/float.c b/src/backend/utils/adt/float.c index 362c29ab803..161d5593f5f 100644 --- a/src/backend/utils/adt/float.c +++ b/src/backend/utils/adt/float.c @@ -24,6 +24,7 @@ #include "common/shortest_dec.h" #include "libpq/pqformat.h" #include "utils/array.h" +#include "utils/builtins.h" #include "utils/float.h" #include "utils/fmgrprotos.h" #include "utils/sortsupport.h" @@ -360,17 +361,18 @@ Datum float4out(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - char *ascii = (char *) palloc(32); + char ascii[32]; int ndig = FLT_DIG + extra_float_digits; if (extra_float_digits > 0) - { float_to_shortest_decimal_buf(num, ascii); - PG_RETURN_CSTRING(ascii); - } + else + (void) pg_strfromd(ascii, 32, ndig, num); - (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(ascii); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -391,11 +393,21 @@ Datum float4send(PG_FUNCTION_ARGS) { float4 num = PG_GETARG_FLOAT4(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat4(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat4_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat4(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -563,8 +575,18 @@ Datum float8out(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); + char ascii[32]; + int ndig = DBL_DIG + extra_float_digits; + + if (extra_float_digits > 0) + double_to_shortest_decimal_buf(num, ascii); + else + (void) pg_strfromd(ascii, 32, ndig, num); - PG_RETURN_CSTRING(float8out_internal(num)); + if (pg_send_inout_text(fcinfo, ascii, strlen(ascii))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(ascii)); } /* @@ -608,11 +630,21 @@ Datum float8send(PG_FUNCTION_ARGS) { float8 num = PG_GETARG_FLOAT8(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendfloat8(&buf, num); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendfloat8_field(buf, num); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendfloat8(&buf, num); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/backend/utils/adt/int.c b/src/backend/utils/adt/int.c index de54a43c98d..9a3fda0403a 100644 --- a/src/backend/utils/adt/int.c +++ b/src/backend/utils/adt/int.c @@ -74,10 +74,27 @@ Datum int2out(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - char *result = (char *) palloc(7); /* sign, 5 digits, '\0' */ + int maxlen = 7; /* sign, 5 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pg_itoa(arg1, result); - PG_RETURN_CSTRING(result); + if (buf) + { + int offset; + int len; + + len = pg_itoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char *result = (char *) palloc(maxlen); + + pg_itoa(arg1, result); + PG_RETURN_CSTRING(result); + } } /* @@ -98,11 +115,21 @@ Datum int2send(PG_FUNCTION_ARGS) { int16 arg1 = PG_GETARG_INT16(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint16(&buf, arg1); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint16_field(buf, arg1); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint16(&buf, arg1); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } /* @@ -328,40 +355,22 @@ int4out(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); int maxlen = 12; /* sign, 10 digits, '\0' */ + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - /* - * Optimized path for output functions called as part of a larger - * ouput. - * - * FIXME: A good chunk of this should obviously be in helper - * functions. - */ - InOutContext *inout = castNode(InOutContext, fcinfo->context); - StringInfo buf = inout->buf; - int prev_buflen; + /* Optimized path for output functions called as part of a larger output. */ + int offset; int len; - uint32 len_net; - - /* reserve space for length and the max string length */ - enlargeStringInfo(buf, sizeof(uint32) + maxlen); - - /* reserve space for length, to be filled out later */ - prev_buflen = buf->len; - buf->len += sizeof(uint32); /* * Construct string directly in buffer, we don't have to care about * encoding conversions, because we assume that every encoding * embodies ascii (XXX: Is that actually true with client encodings?). */ - len = pg_ltoa(arg1, buf->data + buf->len); + len = pg_ltoa(arg1, pq_begincountedfield(buf, maxlen, &offset)); buf->len += len; - - /* update the previously reserved length */ - len_net = pg_hton32(len); - memcpy(&buf->data[prev_buflen], &len_net, sizeof(uint32)); + pq_endcountedfield(buf, offset); PG_RETURN_VOID(); } @@ -395,16 +404,11 @@ Datum int4send(PG_FUNCTION_ARGS) { int32 arg1 = PG_GETARG_INT32(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - InOutContext *inout = castNode(InOutContext, fcinfo->context); - - /* length of data */ - pq_sendint32(inout->buf, 4); - /* data itself */ - pq_sendint32(inout->buf, arg1); - + pq_sendint32_field(buf, arg1); PG_RETURN_VOID(); } else diff --git a/src/backend/utils/adt/int8.c b/src/backend/utils/adt/int8.c index 9b429da86d9..184927f4438 100644 --- a/src/backend/utils/adt/int8.c +++ b/src/backend/utils/adt/int8.c @@ -63,19 +63,37 @@ Datum int8out(PG_FUNCTION_ARGS) { int64 val = PG_GETARG_INT64(0); - char buf[MAXINT8LEN + 1]; - char *result; - int len; + int maxlen = MAXINT8LEN + 1; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - len = pg_lltoa(val, buf) + 1; + if (buf) + { + int offset; + int len; - /* - * Since the length is already known, we do a manual palloc() and memcpy() - * to avoid the strlen() call that would otherwise be done in pstrdup(). - */ - result = palloc(len); - memcpy(result, buf, len); - PG_RETURN_CSTRING(result); + len = pg_lltoa(val, pq_begincountedfield(buf, maxlen, &offset)); + buf->len += len; + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + char buf[MAXINT8LEN + 1]; + char *result; + int len; + + len = pg_lltoa(val, buf) + 1; + + /* + * Since the length is already known, we do a manual palloc() and + * memcpy() to avoid the strlen() call that would otherwise be done in + * pstrdup(). + */ + result = palloc(len); + memcpy(result, buf, len); + PG_RETURN_CSTRING(result); + } } /* diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c index cb23dfe9b95..1264a0bb0ff 100644 --- a/src/backend/utils/adt/numeric.c +++ b/src/backend/utils/adt/numeric.c @@ -808,11 +808,16 @@ numeric_out(PG_FUNCTION_ARGS) if (NUMERIC_IS_SPECIAL(num)) { if (NUMERIC_IS_PINF(num)) - PG_RETURN_CSTRING(pstrdup("Infinity")); + str = "Infinity"; else if (NUMERIC_IS_NINF(num)) - PG_RETURN_CSTRING(pstrdup("-Infinity")); + str = "-Infinity"; else - PG_RETURN_CSTRING(pstrdup("NaN")); + str = "NaN"; + + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + + PG_RETURN_CSTRING(pstrdup(str)); } /* @@ -822,6 +827,9 @@ numeric_out(PG_FUNCTION_ARGS) str = get_str_from_var(&x); + if (pg_send_inout_text(fcinfo, str, strlen(str))) + PG_RETURN_VOID(); + PG_RETURN_CSTRING(str); } @@ -1147,21 +1155,37 @@ numeric_send(PG_FUNCTION_ARGS) { Numeric num = PG_GETARG_NUMERIC(0); NumericVar x; - StringInfoData buf; + StringInfo buf; + StringInfoData localbuf; + bool inout; int i; init_var_from_num(num, &x); - pq_begintypsend(&buf); + buf = pg_get_inout_context_buf(fcinfo); + inout = buf != NULL; + if (inout) + { + /* length of data */ + pq_sendint32(buf, (4 + x.ndigits) * sizeof(int16)); + } + else + { + pq_begintypsend(&localbuf); + buf = &localbuf; + } - pq_sendint16(&buf, x.ndigits); - pq_sendint16(&buf, x.weight); - pq_sendint16(&buf, x.sign); - pq_sendint16(&buf, x.dscale); + pq_sendint16(buf, x.ndigits); + pq_sendint16(buf, x.weight); + pq_sendint16(buf, x.sign); + pq_sendint16(buf, x.dscale); for (i = 0; i < x.ndigits; i++) - pq_sendint16(&buf, x.digits[i]); + pq_sendint16(buf, x.digits[i]); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (inout) + PG_RETURN_VOID(); + else + PG_RETURN_BYTEA_P(pq_endtypsend(&localbuf)); } diff --git a/src/backend/utils/adt/timestamp.c b/src/backend/utils/adt/timestamp.c index a20e7ea1d11..0c0b7af700a 100644 --- a/src/backend/utils/adt/timestamp.c +++ b/src/backend/utils/adt/timestamp.c @@ -227,7 +227,6 @@ Datum timestamp_out(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - char *result; struct pg_tm tt, *tm = &tt; fsec_t fsec; @@ -242,8 +241,10 @@ timestamp_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -286,11 +287,21 @@ Datum timestamp_send(PG_FUNCTION_ARGS) { Timestamp timestamp = PG_GETARG_TIMESTAMP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum @@ -773,7 +784,6 @@ Datum timestamptz_out(PG_FUNCTION_ARGS) { TimestampTz dt = PG_GETARG_TIMESTAMPTZ(0); - char *result; int tz; struct pg_tm tt, *tm = &tt; @@ -790,8 +800,10 @@ timestamptz_out(PG_FUNCTION_ARGS) (errcode(ERRCODE_DATETIME_VALUE_OUT_OF_RANGE), errmsg("timestamp out of range"))); - result = pstrdup(buf); - PG_RETURN_CSTRING(result); + if (pg_send_inout_text(fcinfo, buf, strlen(buf))) + PG_RETURN_VOID(); + else + PG_RETURN_CSTRING(pstrdup(buf)); } /* @@ -835,11 +847,21 @@ Datum timestamptz_send(PG_FUNCTION_ARGS) { TimestampTz timestamp = PG_GETARG_TIMESTAMPTZ(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendint64(&buf, timestamp); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + pq_sendint64_field(buf, timestamp); + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendint64(&buf, timestamp); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } Datum diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c index 4948ced7dec..5dac5b39f11 100644 --- a/src/backend/utils/adt/varlena.c +++ b/src/backend/utils/adt/varlena.c @@ -289,37 +289,15 @@ Datum textout(PG_FUNCTION_ARGS) { Datum txt = PG_GETARG_DATUM(0); + StringInfo buf = pg_get_inout_context_buf(fcinfo); - if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + if (buf) { - StringInfo buf = castNode(InOutContext, fcinfo->context)->buf; text *tunpacked = pg_detoast_datum_packed(DatumGetPointer(txt)); int len = VARSIZE_ANY_EXHDR(tunpacked); char *data = VARDATA_ANY(tunpacked); - char *data_converted; - size_t data_len; - - /* - * Convert text output to the right encoding. For efficiency, this - * should really happen directly into buf. For that we would have to - * reserve space for the length first and fill it out after - * conversion. - * - * FIXME: Obviously we would need helpers for this too. - */ - data_converted = pg_server_to_client(data, len); - - if (data == data_converted) - data_len = len; - else - data_len = strlen(data_converted); - - /* length */ - pq_sendint32(buf, data_len); - - /* actual data */ - appendBinaryStringInfoNT(buf, data_converted, data_len); + pq_sendcountedtext(buf, data, len); if (tunpacked != DatumGetPointer(txt)) pfree(tunpacked); @@ -356,11 +334,27 @@ Datum textsend(PG_FUNCTION_ARGS) { text *t = PG_GETARG_TEXT_PP(0); - StringInfoData buf; + StringInfo buf = pg_get_inout_context_buf(fcinfo); - pq_begintypsend(&buf); - pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); - PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + if (buf) + { + int offset; + + /* reserve space for length, to be filled out after conversion */ + (void) pq_begincountedfield(buf, 0, &offset); + pq_sendtext(buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + pq_endcountedfield(buf, offset); + + PG_RETURN_VOID(); + } + else + { + StringInfoData buf; + + pq_begintypsend(&buf); + pq_sendtext(&buf, VARDATA_ANY(t), VARSIZE_ANY_EXHDR(t)); + PG_RETURN_BYTEA_P(pq_endtypsend(&buf)); + } } diff --git a/src/include/libpq/pqformat.h b/src/include/libpq/pqformat.h index bc4ab1381a9..d40f36b5fbc 100644 --- a/src/include/libpq/pqformat.h +++ b/src/include/libpq/pqformat.h @@ -94,6 +94,32 @@ pq_writeint64(StringInfoData *pg_restrict buf, uint64 i) buf->len += sizeof(uint64); } +/* + * Reserve a length-prefixed field in a StringInfo buffer, returning the + * location at which the payload should be written. The payload length is + * filled in by pq_endcountedfield(). + */ +static inline char * +pq_begincountedfield(StringInfo buf, int maxlen, int *offset) +{ + enlargeStringInfo(buf, sizeof(uint32) + maxlen); + + *offset = buf->len; + buf->len += sizeof(uint32); + + return buf->data + buf->len; +} + +/* Fill in the length of a field started by pq_begincountedfield(). */ +static inline void +pq_endcountedfield(StringInfo buf, int offset) +{ + int len = buf->len - offset - sizeof(uint32); + uint32 len_net = pg_hton32(len); + + memcpy(&buf->data[offset], &len_net, sizeof(uint32)); +} + /* * Append a null-terminated text string (with conversion) to a buffer with * preallocated space. @@ -187,6 +213,41 @@ pq_sendint(StringInfo buf, uint32 i, int b) } } +/* append length-prefixed binary fields to a StringInfo buffer */ +static inline void +pq_sendint16_field(StringInfo buf, uint16 i) +{ + pq_sendint32(buf, sizeof(uint16)); + pq_sendint16(buf, i); +} + +static inline void +pq_sendint32_field(StringInfo buf, uint32 i) +{ + pq_sendint32(buf, sizeof(uint32)); + pq_sendint32(buf, i); +} + +static inline void +pq_sendint64_field(StringInfo buf, uint64 i) +{ + pq_sendint32(buf, sizeof(uint64)); + pq_sendint64(buf, i); +} + +static inline void +pq_sendfloat4_field(StringInfo buf, float4 f) +{ + pq_sendint32(buf, sizeof(float4)); + pq_sendfloat4(buf, f); +} + +static inline void +pq_sendfloat8_field(StringInfo buf, float8 f) +{ + pq_sendint32(buf, sizeof(float8)); + pq_sendfloat8(buf, f); +} extern void pq_begintypsend(StringInfo buf); extern bytea *pq_endtypsend(StringInfo buf); diff --git a/src/include/utils/builtins.h b/src/include/utils/builtins.h index b6a11bfa288..0bf228a746b 100644 --- a/src/include/utils/builtins.h +++ b/src/include/utils/builtins.h @@ -15,12 +15,36 @@ #define BUILTINS_H #include "fmgr.h" +#include "libpq/pqformat.h" +#include "nodes/miscnodes.h" #include "nodes/nodes.h" #include "utils/fmgrprotos.h" /* Sign + the most decimal digits an 8-byte number could have */ #define MAXINT8LEN 20 +/* Helpers for output/send functions using InOutContext. */ +static inline StringInfo +pg_get_inout_context_buf(FunctionCallInfo fcinfo) +{ + if (fcinfo->context && IsA(fcinfo->context, InOutContext)) + return castNode(InOutContext, fcinfo->context)->buf; + + return NULL; +} + +static inline bool +pg_send_inout_text(FunctionCallInfo fcinfo, const char *str, int slen) +{ + StringInfo buf = pg_get_inout_context_buf(fcinfo); + + if (!buf) + return false; + + pq_sendcountedtext(buf, str, slen); + return true; +} + /* bool.c */ extern bool parse_bool(const char *value, bool *result); extern bool parse_bool_with_len(const char *value, size_t len, bool *result); -- 2.43.0 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=v2-0004-Let-pgbench-support-binary-format.patch ^ permalink raw reply [nested|flat] 513+ messages in thread
end of thread, other threads:[~2026-06-02 09:37 UTC | newest] Thread overview: 513+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-05-24 02:26 [PATCH v20 6/8] Row pattern recognition patch (docs). Tatsuo Ishii <ishii@postgresql.org> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com> 2026-06-02 09:37 [PATCH v2 3/4] Optimize more data type for less memory copy with optional context. Andy Fan <zhihuifan1213@163.com>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox